New Algorithms for Optimizing Multi-Class Classifiers via ROC Surfaces
نویسندگان
چکیده
We study the problem of optimizing a multiclass classifier based on its ROC hypersurface and a matrix describing the costs of each type of prediction error. For a binary classifier, it is straightforward to find an optimal operating point based on its ROC curve and the relative cost of true positive to false positive error. However, the corresponding multiclass problem (finding an optimal operating point based on a ROC hypersurface and cost matrix) is more challenging. We present several heuristics for this problem, including linear and nonlinear programming formulations, genetic algorithms, and a customized algorithm. Empirical results suggest that genetic algorithms fare the best overall, improving performance most often.
منابع مشابه
Noncost Sensitive SVM Training Using Multiple Model Selection
In this paper, we propose a multi-objective optimization framework for SVM hyperparameters tuning. The key idea is to manage a population of classifiers optimizing both False Positive (FP) and True Positive (TP) rates rather than a single classifier optimizing a scalar criterion. Hence, each classifier in teh population optimizes a particular trade-off between the objectives. Within the context...
متن کاملPosition Paper: Extensions of ROC Analysis to Multi-Class Domains
Receiver-operating characteristic (ROC) analysis has proven to be a powerful method for dealing with misclassification costs and skewed class distributions (Provost & Fawcett, 1998). In the typical representation, an ROC analysis evaluates false accept versus false reject rates for a set of candidate binary classifiers under all possible (linear) cost and prior class distribution assumptions. T...
متن کاملFormulation and comparison of multi-class ROC surfaces
The Receiver Operating Characteristic (ROC) has become a standard tool for the analysis and comparision of classifiers when the costs of misclassification are unknown. There has been relatively little work, however, examining ROC for more than two classes. Here we define the ROC surface for the Qclass problem in terms of a multi-objective optimisation problem in which the goal is to simultaneou...
متن کاملVisualisation of multi-class ROC surfaces
The Receiver Operating Characteristic (ROC) has become a standard tool for the analysis and comparison of binary classifiers when the costs of misclassification are unknown. Although there has been relatively little work in examining ROC for more than two classes – there has been growing interest in the area, and in recent studies we have formulated it in terms of misclassification rates. Altho...
متن کاملOptimizing Area Under the ROC Curve using Ranking SVMs
Area Under the ROC Curve (AUC), often used for comparing classifiers, is a widely accepted performance measure for ranking instances. Many researches have studied optimization of AUC, usually via optimizing some approximation of a ranking function. Ranking SVMs are among the better performers but their usage in the literature is typically limited to learning a total ranking from partial ranking...
متن کامل